System For Generating Tag Layouts

ABSTRACT

Generating a tag layout from a set of tags and an ordering of the set of tags, wherein each tag includes a text label and a size for the text label, is disclosed. The system includes a processor accessible memory for receiving an ordered set of tags, each tag including a text label and a size for the text label, and at least one closed shape corresponding to a space for the tag layout. The system further includes a processor for generating the tag layout by computing a scale factor for either the closed shape or the size of the text labels in the set of tags such that all the tags in the set of tags fit within the closed shape, and the processor stores the generated tag layout in the memory.

CROSS REFERENCE TO RELATED APPLICATIONS

Reference is made to commonly assigned U.S. patent application Ser. No.______ filed concurrently herewith, entitled “Method For Generating TagLayouts”, the disclosure of which is incorporated by reference herein inits entirety.

Reference is made to commonly assigned U.S. patent application Ser. No.______ filed concurrently herewith, entitled “Method For Computing ScaleFor Tag Insertion”, the disclosure of which is incorporated by referenceherein in its entirety.

FIELD OF THE INVENTION

This invention relates to the field of generating layouts of shapes, andmore particularly to a method and system for generating layouts for anordered set of first shapes placed within a second set of closed shapesby appropriate scaling.

BACKGROUND OF THE INVENTION

In recent years, information on the World Wide Web has grown drasticallyand is continuing to do so at an ever increasing rate. The prevalence ofsocial media culture has resulted in the digitization of all aspects oflives (i.e. from conversations to celebrations). In other words, humanlives have largely become synonymous with the information we consume andshare on the Web. Today's man is surrounded by a plethora of informationof various kinds in his disparate digital devices. Ironically, the timeslice devoted for consumption of a given piece of information isdecreasing by the day. Therefore, the need for concise and meaningfulinformation presentation has become critical.

Undoubtedly, text constitutes the most abundant form of information onthe Web. However, the structure of text on the Web can vary fromextremely un-syntactical (e.g. tags or short form sentences) to verystructured (e.g. well written and edited articles). Of late, tag-cloudshave gained significance as a way to visualize structured as well asunstructured text. A tag-cloud is a visual depiction of the word contentof a document. A tag-cloud can provide a quick word-content summary oflarge documents, collections, or tag-sets. It can be constructed bytag-frequencies or derived from an ordered tag-set by using tag-weights.An appealing aspect of tag-clouds is the presentation of the relativeemphasis or importance of different words or concepts in a seeminglysimple manner that a human eye can quickly discern (in contrast tolisting numeric weights against different words).

Prior art in generating layouts of tag-clouds for visualization limitsthe shape of tag layouts or does not preserve the ordering of the set oftags. In addition, prior art for generating tag layouts can result inlayouts with tags repeated to fill the space, or omitted due to lack ofspace, described by the shape instead of scaling the tags or the shapesto achieve a good fit.

SUMMARY OF THE INVENTION

The present invention is directed to producing a combined image of alayout by placing a first set of shapes within a second set of shapeswhile preserving criteria including a certain ordering of the first setof shapes, amount of overlap of the first set of shapes as placed in thesecond set of shapes, or the scaling of the first or second set ofshapes.

According to the present invention, a system for generating a tag layoutcomprises:

a processor accessible memory for receiving a set of tags, each tagincluding a text label and a size for the text label, the set of tagshaving an ordering associated with it, and at least one closed shapecorresponding to a space for the tag layout; and

a processor for generating the tag layout in response to the processoraccessible memory by computing a scale factor for at least one of theclosed shape or the size of the text labels in the set of tags of theset of tags such that all the tags in the set of tags fit within theclosed shape and the tags are placed in the space based at least uponthe ordering of the tags in the set of tags to generate the tag layoutand for storing the generated tag layout in the processor accessiblememory for future use.

An advantage of the present invention is that the generated layout is ofan arbitrary given shape and compact and the ordering of the first setof shapes is preserved in the layout. This provides an improved visualrepresentation of the first set of shapes in the layout.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a high-level diagram showing the components of a system forgenerating a tag layout according to one aspect of the presentinvention;

FIG. 2 is a flowchart showing a method for generating a tag layoutaccording to one aspect of the present invention;

FIG. 3 is a flowchart showing a method for computing a scale factoraccording to one aspect of the present invention;

FIG. 4 is a flowchart showing additional details for step 210 of FIG. 3;

FIG. 5 is a flowchart showing additional details for step 230 of FIG. 3according to one aspect of the present invention;

FIG. 6 is a flowchart showing additional details for step 230 of FIG. 3according to another aspect of the present invention;

FIG. 7 is a flowchart showing additional details for step 435 of FIG. 5and FIG. 6;

FIG. 8 shows an illustration of converting a polygon to a set ofintervals.

FIG. 9 shows an illustration of step 635 of FIG. 7;

FIG. 10 shows examples of inserting a set of tags into a polygon usingvarious scale factors;

FIG. 11 shows an example of a set of tags with associated enclosing setof rectangles;

FIG. 12 shows examples of generated tag-layouts; and

FIG. 13 shows an example of a print with a generated tag-layout.

It is to be understood that the attached drawings are for purposes ofillustrating aspects of the invention and may not be to scale.

DETAILED DESCRIPTION OF THE INVENTION

The present invention is directed to producing a combined image showinga layout generated by placing a first set of shapes within a second setof shapes while preserving criteria including a certain ordering of thefirst set of shapes, amount of overlap of the first set of shapes asplaced in the second set of shapes, or the scaling of the first orsecond set of shapes. According to an aspect of the present invention,the first set of shapes can be a set of tags, with an orderingassociated with the tags, the second set of shapes can be a set ofclosed shapes or polygons, and the combined image can be a generated taglayout. FIG. 1 illustrates an electronic system 26, a computer system,for implementing certain aspects of the present invention for generatingtag layouts. In the embodiment of FIG. 1, electronic computer system 26comprises a housing 22 and a source of content and program data files 24such as software applications, image files, sets of tags, ordering oftags, and closed shapes, which includes processor accessible memory 40,a wired user input system 68 as well as an online printing service 58,and an output system 28, all communicating directly or indirectly withprocessor 34. Although not shown processor 34 is meant to illustratetypical processor system and chip components such as instruction andexecution registers, an ALU, various levels of cache memory, etc. Thesource of program and content data files 24, user input system 68, oroutput system 28, and processor 34 can be located within housing 22 asillustrated. Circuits and systems of the source of content and programdata files 24, user input system 68 or output system 28 can be locatedin whole or in part outside of housing 22. As an example, element 68 billustrates a screen pointer control embodied as a mouse when locatedoutside the housing 22 but can be an embedded trackball when locatedwithin housing 22.

The source of content or program data files 24 can include any form ofelectronic, optical, or magnetic storage such as optical discs, storagediscs, diskettes, flash drives, etc., or other circuit or system thatcan supply digital data to processor 34 from which processor 34 can loadsoftware applications, image files, sets of tags, ordering of tags, andclosed shapes or receive software applications, image files, sets oftags, ordering of tags, and closed shapes required to generated a taglayout. In this regard, the content and program data files can comprise,for example and without limitation, software applications, a still imagedata base, image sequences, a video data base, graphics, and computergenerated images, sets of tags, ordering of tags, closed shapes and anyother data necessary for practicing aspects of the present invention asdescribed herein. Source of content data files 24 can optionally includedevices to capture images to create content data for use in content datafiles by use of capture devices and/or can obtain content data filesthat have been prepared by or using other devices or image enhancementand editing software. In FIG. 1, sources of content or program datafiles 24 include sensors 38, processor accessible memory 40, and acommunication system 54.

Sensors 38 are optional for particular aspects of the present inventionand can include light sensors, biometric sensors, and other sensorsknown in the art that can be used to detect conditions in theenvironment of system 26 and to convert this information into a formthat can be used by processor 34 of system 26. Sensors 38 can alsoinclude one or more cameras, video sensors, scanners, microphones, PDAs,palm tops, laptops that are adapted to capture images and can be coupledto processor 34 directly by cable or by removing portable memory 39 fromthese devices and/or computer systems and coupling the portable memoryto slot 46. Sensors 38 can also include biometric or other sensors formeasuring involuntary physical and mental reactions. Such sensorsincluding, but not limited to, voice inflection, body movement, eyemovement, pupil dilation, body temperature, and p4000 wave sensors.

Processor accessible memory 40 can include conventional digital memorydevices including solid state, magnetic, optical or other data storagedevices, as mentioned above. Processor accessible memory 40 can be fixedwithin system 26 or it can be removable and portable. In FIG. 1, system26 is shown having a hard disk drive 42, which can be an attachableexternal hard drive, which can include an operating system forelectronic computer system 26, and other software programs andapplications such as the program algorithm aspects of the presentinvention, sets of tags, ordering of tags, closed shapes, softwareapplications, and a digital image data base. A disk drive 44 for aremovable disk such as an optical, magnetic or other disk memory (notshown) can also include control programs and software programs usefulfor certain embodiments of the present invention, and a memory card slot46 that holds a removable portable memory 48 such as a removable memorycard or flash memory drive or other connectable memory and has aremovable memory interface 50 for communicating with removable memory48, if necessary. Data including, but not limited to, control programs,image files, sets of tags, ordering of tags, closed shapes, softwareapplications, digital images, and metadata can also be stored in aremote memory system 52 such as a personal computer, computer network, anetwork connected server, or other digital system.

In FIG. 1, system 26 has a communication system 54 that can be used tocommunicate with an online printing service 58, remote memory system 52,or remote display 56, for example by transmitting generated tag layoutsand receiving from remote memory system 52, a variety of controlprograms, software applications, image files, sets of tags, ordering oftags, and closed shapes. Although communication system 54 is shown as awireless communication system, it can also include a modem for couplingto a network over a communication cable for providing to the computersystem 26 access to the network and remote memory system 52. An onlineprinting service 58 including an online server 56 and/or remote inputcontrols 58 a, 58 b, or 58 c can communicate with communication system54 wirelessly as illustrated or, again, can communicate in a wiredfashion. In a preferred embodiment, a local input station includingeither or both of a local display 66 and local user input controls 68(also referred to herein as “local user input 68”) is connected toprocessor 34 which is connected to communication system 54 using a wiredor wireless connection.

Communication system 54 can comprise for example, one or more optical,radio frequency or other transducer circuits or other systems thatconvert data into a form that can be conveyed to a remote device such asremote memory system 52 or remote display 56 using an optical signal,radio frequency signal or other form of signal. Communication system 54can also be used to receive a digital image and other data, asexemplified above, from a host or server computer or network (notshown), a remote memory system 52 or an online printing service 58.Communication system 54 provides processor 34 with information andinstructions from signals received thereby. Typically, communicationsystem 54 will be adapted to communicate with the remote memory system52 by way of a communication network such as a conventionaltelecommunication or data transfer network such as the internet, acellular, peer-to-peer or other form of mobile telecommunicationnetwork, a local communication network such as wired or wireless localarea network or any other conventional wired or wireless data transfersystem.

User input system 68 provides a way for a user of system 26 to provideinstructions to processor 34, such instructions comprising automatedsoftware algorithms of particular aspects of the present invention thatgenerate tag layouts. This software also allows a user to make adesignation of content data files, such as sets of tags, closed shapes,and ordering of tags, to be used in generating a tag layout according toan aspect of the present invention and to select an output form for theoutput product. User controls 68 a, 68 b or 58 a, 58 b in user inputsystem 68 and online printing service 58, respectively, can also be usedfor a variety of other purposes including, but not limited to, allowinga user to arrange, organize and edit content data files, such as imagefiles, sets of tags, ordering of tags, and closed shapes to be used togenerate the tag layout, for example, by incorporating image editingsoftware in computer system 26 which can be used to edit tag layoutgenerated by computer system 26, to provide information about the useror audience, to provide annotation data such as voice and text data, toidentify characters in the content data files, and to perform otherinteractions with system 26.

In this regard user input system 68 can comprise any form of devicecapable of receiving an input from a user and converting this input intoa form that can be used by processor 34. For example, user input system68 can comprise a touch screen input 66, a touch pad input, a 4-wayswitch, a 6-way switch, an 8-way switch, a stylus system, a trackballsystem, a joystick system, a voice recognition system, a gesturerecognition system, a keyboard 68 a, mouse 68 b, a remote control orother such systems. In FIG. 1, electronic computer system 26 includes anonline printing system 58 including, but not limited to, a remotekeyboard 58 a, a remote mouse 58 b, and a remote control 58 c.Similarly, local input 68 can take a variety of forms. In FIG. 1, localdisplay 66 and local user input 68 are shown directly connected toprocessor 34.

FIG. 2 shows a flowchart for generating the tag layout according to oneaspect of the present invention. It will be understood that thisflowchart, and the other flowcharts described hereafter, arerepresentative of algorithms practiced by the processor 34 shown inFIG. 1. In step 100 of FIG. 2, the set of tags 205 is received. Each tagin the set of tags 205 includes a text label and a size for the textlabel. The size of the text label can be represented as a font size fordisplaying the text label. The size of the text label can be determinedby counting the frequency of occurrence of the tag in a set ofdocuments, image metadata, or labels. An ordering of the set of tags 205is received in step 105. This ordering can be alphabetical order orgenerated using semantic analysis. Next, at least one closed shape,corresponding to a space for the tag layout 245, is received in step110. In 2-D space, such a shape can be represented by a set of polygons200 of arbitrary shape. In addition, each polygon 800 in the set ofpolygons 200 can be separate from each other or connected to each other.From this point on in this text, the terms closed shape, set of polygons200, and polygon 800 will be used interchangeably. In step 115,processor 34 is used to compute a scale factor 235 for at least one ofthe polygons 800 in the set of polygons 200 or the size of the textlabels in the set of tags 205. In step 120, the tag layout 245 isgenerated. All the tags in the set of tags 205 fit within the set ofpolygons 200 and there is no overlap between the tags as laid out in thegenerated tag layout 245. The tags are placed in the space based atleast upon the ordering of the tags in the set of tags 205. Thegenerated tag layout 245 is stored in processor accessible memory 40 instep 125.

FIG. 3 shows a flowchart for computing the scale factor 235 according tostep 115 of FIG. 2. The set of polygons 200 is provided as input to step210. Step 210 converts the set of polygons 200 to a set of intervals220. The details of step 210 are shown in FIG. 4 and will be describedlater in the specification. The set of tags 205 is provided as input tostep 215. In step 215, an enclosing set of rectangles 225 is determined.The number of tags in the set of tags 205 is represented as N. Thesetags can have different font sizes. The tags are represented by a set ofrectangles 225 denoted by T, where T={T_(k)|0≦k≦N−1}. The width w_(k)and height h_(k) of a rectangle T_(k), are set by the length and heightof the k+1^(th) tag so that the rectangle T_(k) can enclose the k+1^(th)tag compactly. The first rectangle is labeled T₀ and the last rectangleis labeled T_(N-1). In various aspects of the present invention, anamount of overlap between the tags in the tag layout 245 can becontrolled using empty boundary space around each tag in the set of tags205. The set of intervals 220 is provided to step 230 to determine thescale factor 235.

In one aspect of the present invention, step 115 can be performediteratively to generate the tag layout 245. In this case, the processor34 can compute the scale factor 235 for the polygon 800 that cancompactly accommodate the ordered set of tags such that the followingcriteria are adhered to.

The layout preserves the order of tags.

Different tags do not overlap within the polygon.

In some aspects of the present invention, the generated tag layout 245can preserve the order of tags going left-right or top-bottom as shownin FIG. 12 and described later in the specification. In other aspects ofthe present invention, the order of tags is not strictly preserved andoverlap of different tags can be customized depending on a user'spreference. Tags are represented by rectangles whose dimensions can bedetermined by their font size and word length. Therefore the layoutproblem is synonymous with the problem of fitting multiple sizedrectangles within an arbitrary shaped polygon and preserving a certainorder in the layout. FIG. 11 shows an example of tags within the set oftags 205 and the enclosing set of rectangles 225. The tag “BIRTHDAY” hasthe largest size, therefore, the corresponding rectangle that fullyencloses this tag is the largest in size. Other rectangles that fullyenclose the other tags are sized appropriate to the sizes of thecorresponding tags.

To aid understanding of step 210, FIGS. 4 and 8 will be simultaneouslydescribed. FIG. 4 shows step 210 of converting the set of polygons 200to the set of intervals 220. The set of polygons 200 can be converted toa set of sequences of integer points 310 using step 305. The set ofsequences of integer points 310 is converted to a set of line segments320 using step 315. The set of line segments 320 is converted to the setof intervals 220 using step 325. FIG. 8 shows the polygon 800, from theset of polygons 200, the set of line segments 320, and the converted setof intervals 220 in accordance with an aspect of the present invention.

In step 315, the set of sequences of integer points 310 denoted by S isconverted to the set of line segments 320. Equation 1 can be used torepresent the polygon 800 as the set of sequence of integer points 310using n points.

S={p _(i)=(x _(i) ,y _(i))|0≦i<n}  (1)

where S is the set of sequence of integer points 310, p_(i) is an i^(th)point on the polygon 800, and x_(i) and y_(i) are coordinates of thei^(th) point. The set of sequence of integer points 310 S can beconverted to the set of line segments 320 denoted by L using Equation(2) as illustrated in FIG. 8 and described below.

L={l _(i)= p_(i) p _(j) |0≦i<n, j=(i+1)%(n+1)}  (2)

where l_(i) is a line segment joining neighboring points p_(i) and p_(j)and the % represents a remainder of a division operation where (i+1) isdivided by (n+1). In an aspect of the present invention, there is atleast one sequence of integer points 310 S, and therefore, there is atleast one set of line segments 320 L.

The set of line segments 320 can be converted to the set of intervals220 using step 325. Equation 3 describes a set of horizontal lines L_(H)at a plurality of integer locations y.

L _(H) ={y=c|┌y _(min) ┐≦c≦└y _(max)┘}  (3)

where y_(min) and y_(max) are a minimum and a maximum value of y_(i)respectively, ┌y_(min)┐ and └y_(max)┘ mean mathematical ceiling andflooring operations on y_(min) and y_(max) respectively, and c is aninteger value between ┌y_(min)┐ and └y_(max)┘ in one aspect of thepresent invention. However, y_(min), y_(max), and c can be real numbersin other aspects of the invention. Next, the intersections between theset of line segments 320 and L_(H) are determined. As illustrated inFIG. 8, a horizontal line at a location where y=c has 4 intersectionswith lines l₁, l₂, l₄, and l₉. A subset of intervals at a particularlocation 805 is produced as a result of these intersections.

The set of intervals 220 denoted by R is constructed as described inEquation 4.

R={r _(c)(i)=[s _(c)(i),e _(c)(i)]|┌y _(min) ┐≦c≦└y _(max)┘,0≦i≦M−1}  (4)

where r_(c)(i) is an i^(th) interval at location c, s_(c)(i) is astarting location of the interval r_(c)(i), e_(c)(i) is an endinglocation of the interval r_(c)(i), M is the number of unique intervalsdetermined by the intersections of L_(H) and the set of line segments320. For example, FIG. 8 shows location y=c has intersections with l₁,l₂, l₄, and l₉ and produces two intervals r_(c)(0)=[s_(c)(0),e_(c)(0)]and r_(c)(1)=[s_(c)(1), e_(c)(1)]. For the set of sequences of integerpoints 310 having more than one sequence, corresponding to the set ofpolygons 200 having more than one polygon 800, each sequence of integerpoints will be processed in the same manner described above to producemultiple sets of intermediate intervals. The set of intervals 220 isproduced by a mathematical set union operation on all of the multiplesets of intermediate intervals.

FIG. 5 shows a flowchart describing additional details for step 230. Aneffective aspect of the present invention is that the computed scalefactor 235 generates the tag layout 245 that has a good fit for the setof tags 205 into the set of polygons 200. This problem can be formulatedas a root-finding problem where the space of scale factor 235 values issearched for possible solutions. Those skilled in the art willappreciate that search methods such as bisection and secant search canbe employed to solve the root-finding problem.

The flow chart of FIG. 5 is summarized first and details of each stepare described later in the specification. An initial scale factor 415,which is denoted by s⁽⁰⁾, is computed in step 410 using the set ofrectangles 225 and the set of intervals 220 as inputs. Next, therectangle T_(k) is inserted into the polygon 800 at initial scale factor415 s⁽⁰⁾. The polygon 800 at initial scale factor 415 s⁽⁰⁾ can berepresented as R×s⁽⁰⁾. Equation 5 describes how to construct the polygon800 R×s⁽⁰⁾.

R×s ⁽⁰⁾ ={r _(c)(i)=[(s _(c)(i)×s ⁽⁰⁾*,(e _(c)(i)×s ⁽⁰⁾)*]}  (5)

where c is in an interval |y_(min)×s⁽⁰⁾|≦c≦└y_(max)×s⁽⁰⁾┘ and *represents a mathematical rounding operation to the closest integer. Thescale factor 235 s is increased when all of the rectangles T_(k) are notinserted into the polygon R×s and the scale factor 235 s is reduced whenthere is a residual space after the successful insertion of all of therectangles T_(k) into the polygon R×s.

Equation 6 describes how to compute a residual area 675 denoted by A_(r)⁺(R′) if all of the rectangles T_(k) are inserted into an updated set ofintervals R′.

$\begin{matrix}{{A_{r}^{+}\left( R^{\prime} \right)} = {\sum\limits_{i = 0}^{M - 1}\; {\sum\limits_{c = {\lceil y_{\min}\rceil}}^{\lfloor y_{\max}\rfloor}\; \left( {{e_{c}^{\prime}(i)} - {s_{c}^{\prime}(i)} + 1} \right)}}} & (6)\end{matrix}$

where R′, e_(c)′(i), and s_(c)′(i) denote updated values of R, e_(c)(i),and s_(c)(i) after the insertion of the set of rectangles 225 into theset of intervals 220, respectively. To aid in understanding, an aspectof the present invention is now described with reference to FIG. 10.FIG. 10 shows examples when the polygon 800 is scaled using variousscale factors to insert the set of rectangles 225 into the set ofintervals 220. FIG. 10 shows a result 1005 of inserting the set ofrectangles 225 at a particular scale factor, a result 1015 with anotherparticular scale factor, and a result 1020 with yet another particularscale factor. The example results 1005, 1015, and 1020 also show theresidual area 675 A_(r) ⁺(R′) left after insertion of the set ofrectangles 225 into the polygon 800 as empty blank space.

If only v number of rectangles T_(k), where v is less than the totalnumber of rectangles N in the set of rectangles 225, are inserted intothe set of intervals 220, an overflow area 670 denoted by A_(r) ⁻(v) canbe computed as described in Equation 7.

$\begin{matrix}{{A_{r}^{-}(v)} = {\sum\limits_{i = v}^{N - 1}\; {w_{k} \times h_{k}}}} & (7)\end{matrix}$

where w_(k) and h_(k) are the width and the height of T_(k),respectively. A result 1000 with a particular scale and a result 1010with another particular scale are shown in FIG. 10. In these results,some of the rectangles T_(k) were not inserted into the set of intervals220 and are shown as uninserted rectangles 1030. Result 1025 shows thefinal generated tag layout 245 at the scale factor 235.

Referring back to FIG. 5, the flowchart shown describes an algorithm forscaling the polygon 800, represented as the set of intervals 220, suchthat the set of rectangles 225 can be inserted into the polygon 800compactly.

Equation 8 describes how to set the initial scale factor 415 denoted bys⁽⁰⁾.

$\begin{matrix}{s^{(0)} = \frac{A_{r}^{-}(0)}{A_{r}^{+}(R)}} & (8)\end{matrix}$

where A_(r) ⁺(R) is computed by setting R′=R in Equation 6 and A⁻_(r)(0) is computed by setting v=0 in Equation 7.

In step 420, variables used to compute the scale factor 235 areinitialized. In one aspect of the present invention, these variablesinclude f representing a goodness of fit 690 of inserting the set ofrectangles 225 into the set of intervals 220 at a particular scalefactor 235 s, s⁺ representing the scale factor 235 that satisfies f>0,f⁺ representing the goodness of fit 690 for s⁺, s⁻ representing thescale factor 235 that satisfies f<0, f⁻ representing the goodness of fit690 for s⁻, and Δs representing a change in scale factor 235. In oneaspect of the present invention, these variables can be initialized asf=1, step=0.5, s=s⁽⁰⁾, s⁺=−1, s⁻=−1, f⁻=0, f⁺=0, and Δs=∞. In step 435,the goodness of fit, f is computed and if there is residual area 680,then s⁺ is set to s and f⁺ is set to f. If f<0, meaning some of the tagswere not inserted into the set of intervals 220, then s⁻ is set to s andf⁻ is set to f. Step 435 is shown in detail in FIG. 7 and will bedescribed later in the specification.

In step 425, the convergence of the root-finding algorithm is checked.If Δs>ε or f<0, the root-finding algorithm has not converged, meaningeither the change in scale factor 235 is greater than a threshold E orsome tags were not inserted into the polygon 800. If Δs≦ε and f≧0, theroot-finding algorithm has converged, meaning all of rectangles T_(k)have been inserted into the set of polygons 200 and the change in scalefactor 235 between a previous iteration and the current iteration issmaller than the threshold ε. In this case, the value of the scalefactor 235 is output to the algorithm described in the flowchart of FIG.3. If the root-finding algorithm has not converged, the next step is440. In step 440, the values of s⁻ and s⁺ are used to compute a boundinginterval. This bounding interval is evaluated to determine if anappropriate scale factor 235 can be found within the bounding interval.If s⁻=−1 or s⁺=−1, the appropriate scale factor 235 is not within thebounding interval and the algorithm resumes at step 445. In step 445,the goodness of fit 690 is checked to determine if there is residualarea 680. If there is residual area, the scale factor 235 is reduced instep 450 and a new goodness of fit 690 is computed. If there is overflowarea 670, the scale factor 235 is increased in step 455 and a newgoodness of fit 690 is computed. If the scale factor 235 is within thebounding interval, the algorithm resumes at step 460. In step 460, a newscale factor s_(new) is computed. The new scale factor can be computedusing methods well known in the art, such as the secant method. Equation9 describes how to compute the new scale factor using the secant method.

$\begin{matrix}{s_{new} = {{{- f^{+}} \times \frac{s^{+} - s^{-}}{f^{+} - f^{-}}} + s^{+}}} & (9)\end{matrix}$

The new scale factor is used to compute a new goodness of fit 690 asdescribed in step 435. This process continues iteratively until theroot-finding algorithm converges as described in step 425.

FIG. 6 is similar to FIG. 5 and where the steps and blocks correspond,they will have the same numbers, but when the functions are different,the steps and blocks will be described below. FIG. 6 shows a flowchartthat describes an algorithm for scaling the set of tags 205, representedas the set of rectangles 225, such that they can be inserted into theset of intervals 220, representing the set of polygons 200, compactlyaccording to another aspect of the present invention. In the algorithmdescribed by the flowchart shown in FIG. 6, steps 550 and 555 aredifferent from FIG. 5. In step 550 the scale factor 235 s of the set ofrectangles 225 is increased since there is residual area present. Instep 555, the scale factor 235 s of the set of rectangles 225 isdecreased since all the rectangles were not inserted.

FIG. 7 shows a flowchart describing an algorithm for computing thegoodness of fit 690. In step 615, the first rectangle T₀ is selected bysetting k=0. Step 615 is followed by decision step 620. In step 620, acheck for uninserted rectangles 1030 is performed. If there areuninserted rectangles 1030, the algorithm resumes with step 625 else thealgorithm resumes with step 675. In step 675, the residual area 680A_(r) ⁺(R′) can be computed using Equation 6, where R′ is theaforementioned updated set of intervals. In a following step 660, theresidual area 680 A_(r) ⁺(R′) is used to determine goodness of fit 690.In step 625, the insertion variables are initialized. In an aspect ofthe present invention, the value of y is initialized to y_(min) in step625. Step 625 is followed by decision step 630. In step 630 the value ofy is checked to determine if an insertion is possible. If y≦y_(max), theinsertion is possible and the algorithm resumes with step 635 else theinsertion is not possible and the algorithm resumes with step 665. Instep 635, a rectangle from the set of rectangles 225 is inserted into asubset of intervals from the set of intervals 220. The details of step635 will be described later in the specification. Step 635 produces aninsertion position and the updated set of intervals 640. As mentionedbefore, the updated set of intervals is denoted by R′. In step 645, theinsertion position denoted by x is checked. If x≠−1, the insertion wassuccessful at 2D integer location (x,y) and the algorithm resumes withstep 650 else the algorithm resumes with step 655 by setting y=y+1 asthe next insertion location. In step 650, the values of k, and R areupdated as k=k+1, and R=R′. In step 665, the overflow area 670 iscomputed by A_(r) ⁻(k) by setting v=k in Equation 7. In the followingstep 660, the overflow area 670 multiplied by −1, that is −1×Ar⁻(k) isused to determine the goodness of fit 690.

In a particular aspect of the present invention, the order is looselymaintained from top left to the bottom right. In other words, the orderof insertion of consecutive ordered rectangles is left to right.However, if there is a rectangle that can be fit into a blank spacebetween two consecutive rectangles in the order specified by theordering of the set of tags 205, the ordering can be ignored locally toachieve a good fit. In other aspects of the invention, the order isstrictly maintained from top left to the bottom right and no insertionof tags is allowed out of order.

The set of rectangles 225 is inserted into the set of intervals 220using step 635 iteratively for each rectangle. As discussed above, step635 preserves the order of the insertion of rectangles from left toright. As the set of intervals 220 is filled with rectangles from theset of rectangles 225, the next rectangle T_(k) is inserted into the setof intervals 220 at the leftmost available 2D integer location (x,y)where y is set either by step 625 or step 655 and x is determined bystep 635.

FIG. 9 shows an illustration of an example of one method for insertingtwo rectangles 900 and 950 into a subset of intervals 905 whilepreserving left-right order. The first rectangle 900 to be inserted intothe subset of intervals 905 has width w₁ and height 3. The rectangle 900is placed at the leftmost available location y=jj in the subset ofintervals 905 as shown in 910. In 915, the rectangle 900 is shifted downand to the right in the subset of intervals 905 at location y=jj+1. Thisrightward shift is performed because the leftmost available insertionlocation at y=jj+1 is at a different location than the leftmostavailable insertion location at y=jj. This process is performediteratively until the rectangle 900 can be completely inserted into thesubset of intervals 905 as shown in 920. After the rectangle 900 hasbeen inserted into the subset of intervals 905, the 2D space occupied bythe rectangle 900 is removed from the subset of intervals 905 and anupdated subset of intervals 930 is formed. The upper left location 925of the inserted rectangle, denoted by ii, and the updated subset ofintervals 930 are used to insert the next rectangle as described in step635.

It is possible that a rectangle 950 having width of W₂ and height 2cannot be inserted into a subset of intervals 955. The rectangle 950 ispositioned at the leftmost available location at y=jj for insertions asshown in 960. The rectangle 950 has to be shifted down to allow forcomplete insertion into the subset of intervals 955. However, as shownin 965, this results in the rectangle 950 extending outside the subsetof intervals 955. In this case, the rectangle 950 cannot be insertedinto the subset of intervals 955 at the current scale and an indicationof this is returned to the algorithm of FIG. 7 by step 635. Thealgorithm changes the scale factor for the set of intervals 220 or theset of rectangles 225 and resumes by attempting to insert all therectangles into the intervals at the changed scale factor.

FIG. 12 shows some examples of generated tag layouts 245. Examplesinclude a heart-shaped tag layout 1200, a star-shaped tag layout 1205,and a circle shaped tag layout 1210. The generated tag layouts can havedifferent color backgrounds. The fonts used for the tags can also bedifferent colored.

FIG. 13 shows a user image 1300 with a sky region boundary 1305corresponding to the closed shape for the tag layout. The generated taglayout 1310 is placed inside the sky region boundary 1305. The user mage1300 with the placed tag layout 1310 can be printed using printer 29 orprovided to online printing service 58 using the communication system 54as shown in FIG. 1.

An aspect of the present invention has the constraint that there is anordering associated with the set of tags 205. In one aspect of thepresent invention, the ordering of the set of tags 205 can be based on asemantic analysis of the set of tags 205. In this case, a co-occurrencematrix can be constructed for the set of tags 205. The co-occurrencematrix measures how often tags appear together in a given set ofpictures or documents. The co-occurrence matrix can be clustered usingmethods well known in the art such as spectral clustering. Theclustering of tags thus obtained can be used to derive an ordering forthe tags such that tags within the same cluster are closer in theordering while tags in different clusters are further in the ordering.In another aspect of the invention, page-rank, as taught in Page et al.,can be computed for each tag based on the co-occurrence matrix. Tags canthen be ordered directly based on their page-rank.

The semantic analysis can also be performed using natural languageunderstanding. Natural language understanding involves classifying tagsinto their respective figures of speech (nouns, adjectives, verbs,adverbs etc.). Such a classification can be performed using WordNet, alexical database of English language developed by the Cognitive ScienceLaboratory at Princeton University. WordNet is a lexical referencesystem that uses psycholinguistic theories of human lexical memory. Thelinguistic classification of tags obtained using WordNet can be used toderive an ordering for the tags such that tags within the samelinguistic class are closer in the ordering while tags in differentlinguistic classes are further in the ordering. In another aspect of theinvention page-rank can be computed for each tag based on a pairwisetags linguistic class matrix (wherein entry for a pair of tags is 1 onlyif they are in the same linguistic class and otherwise the entry is 0).Tags can then be ordered directly based on their page-rank.

In another aspect of the present invention, the ordering of the set oftags 205 can be performed based on user preference. A user's picturecollection can be analyzed to reveal the user's preferences. Pictures ina user's collection can be annotated with words. The image annotationsobtained from user's collection can be used to derive an ordering forthe tags such that tags that are related to the image annotations areranked higher in the ordering and tags that are unrelated to the imageannotations are ranked lower in the ordering.

In another aspect of the present invention, there can be more than oneclosed shapes, represented using the set of polygons 200, for placingthe set of tags 205 to generate the tag layout 245. In this case, theset of tags 205 has to be divided into subsets of tags. The number ofsubsets of tags is equal to the number of polygons 800 in the set ofpolygons 200. In one aspect of the present invention, the set of tags205 is divided into subsets of tags based upon the relative sizes of thepolygons 800 and the sizes of the tags in the set of tags. The sizes ofall of the polygons 800 are added together to generate a total size forthe set of polygons. A subset size for each polygon 800 in the set ofpolygons 200 is determined by computing a ratio between the size of thepolygon 800 and the total size of all the polygons. The set of tags 205is represented by the set of rectangles T 225, where the size of eachrectangle in the set of rectangles 225 is based on the size of thecorresponding tag. The sizes of all the rectangles in the set ofrectangles 225 are added together to compute a total size for therectangles to be placed into the set of polygons 200. The set ofrectangles 225 can be divided into subsets based on the computed ratiosbetween the sizes of the polygons 800 and the total size of all thepolygons. The number of subsets of sets of rectangles 225 is equal tothe number of polygons 800 in the set of polygons 200.

In one aspect of the present invention, the set of tags 205 is splitinto subsets based upon the ordering of the set of tags 205 such apolygon 800 in the set of polygons 200 has a subset of consecutive tagsin the set of tags 205. In this aspect, the ordering is maintainedwithin each polygon 800 independently of other polygons in the set ofpolygons 200. In another aspect of the present invention, the tags canbe split across the polygons 800 such that the ordering is maintainedacross all the polygons simultaneously. In this case, the subset of tagsassociated with a polygon 800 may not have consecutive tags from the setof tags 205.

The invention has been described in detail with particular reference topreferred embodiments thereof, but it will be understood that variationsand modifications can be effected within the spirit and scope of theinvention.

PARTS LIST

-   22 housing-   24 source of data files-   26 system-   28 system-   29 printer-   30 printer-   32 I/O-   34 processor-   35 I/O-   38 sensor-   39 memory-   40 processor accessible memory-   42 storage-   44 storage-   46 memory card slot-   48 memory-   50 interface-   52 memory-   54 communication system-   56 I/O-   58 online printing service-   58 a I/O-   58 b I/O-   58 c I/O-   66 I/O-   68 I/O-   68 a I/O-   68 b I/O-   100 receive a set of tags-   105 receive an ordering of the set of tags-   110 receive at least one closed shape-   115 compute a scale factor-   120 generate a tag layout-   125 store the generated tag layout-   200 set of polygons-   205 set of tags-   210 convert polygons to intervals-   215 determine enclosing set of rectangles-   220 set of intervals-   225 set of rectangles-   230 determine scale factor-   235 scale factor-   245 tag layout-   305 convert to sequences of integer points-   310 set of sequences of integer points-   315 convert to line segments-   320 set of line segments-   325 convert to intervals-   410 determine initial scale factor-   415 initial scale factor-   420 initialize variables-   425 check if converged-   435 compute goodness of fit-   440 check if scale factor bounded-   445 check if residual area-   450 update values to reduce scale factor-   455 update values to increase scale factor-   460 compute new scale factor-   550 update values to increase scale factor-   555 update values to reduce scale factor-   615 select rectangle for insertion-   620 check for uninserted rectangles-   625 initialize insertion variables-   630 check if insertion possible-   635 insert rectangle into interval-   640 insertion position and updated set of intervals-   645 check if inserted-   650 update variables for next insertion-   655 select next insertion location-   660 determine goodness of fit-   665 compute overflow area-   670 overflow area-   675 compute residual area-   680 residual area-   690 goodness of fit-   800 a polygon in the set of polygons-   805 subset of intervals at a particular location-   900 rectangle A-   905 set of intervals A-   910 insertion progress A at y=jj-   915 insertion progress A at y=jj+1-   920 insertion progress A at y=jj+2-   925 final leftmost available horizontal location-   930 updated set of intervals A-   950 rectangle B-   955 set of intervals B-   960 insertion progress B at y=jj-   965 insertion progress B at y=jj+1-   1000 tag layout at a first scale-   1005 tag layout at a second scale-   1010 tag layout at a third scale-   1015 tag layout at a fourth scale-   1020 tag layout at a fifth scale-   1025 tag layout at a sixth scale-   1030 uninserted rectangles-   1200 Heart-shaped tag layout-   1205 Star-shaped tag layout-   1210 Circle shaped tag layout-   1300 user image-   1305 sky region boundary-   1310 tag layout placed in user image

1. A system for generating a tag layout, comprising: a processoraccessible memory for receiving a set of tags, each tag including a textlabel and a size for the text label, the set of tags having an orderingassociated with it, and at least one closed shape corresponding to aspace for the tag layout; and a processor for generating the tag layoutin response to the processor accessible memory by computing a scalefactor for at least one of the closed shape or the size of the textlabels in the set of tags of the set of tags such that all the tags inthe set of tags fit within the closed shape, the tags are placed in thespace based at least upon the ordering of the tags in the set of tags togenerate the tag layout and storing the generated tag layout in theprocessor accessible memory for future use.
 2. The system of claim 1wherein the set of tags is ordered alphabetically.
 3. The system ofclaim 1 wherein the set of tags is ordered based on a semantic analysisof the set of tags.
 4. The system of claim 3 wherein the semanticanalysis is performed using natural language understanding.
 5. Thesystem of claim 1 wherein the set of tags is ordered based on preferenceof a user.
 6. The system of claim 1 wherein the closed shape is apolygon.
 7. The system of claim 1 wherein there are at least two closedshapes further including the processor dividing the set of tags intosubsets of tags, wherein the number of subsets of tags is equal to thenumber of closed shapes, wherein each subset of tags corresponds to aparticular closed shape.
 8. The system of claim 7 wherein the set oftags is divided into subsets of tags based at least upon the relativesizes of the closed shapes and the tags in the set of tags and theordering of the tags in the set of tags.
 9. The system of claim 1wherein the closed shape is user defined.
 10. The system of claim 1wherein the tags are placed in the space corresponding to the closedshape in left to right order followed by top to bottom order.
 11. Thesystem of claim 1 wherein the tags are placed in the space correspondingto the closed shape in top to bottom order followed by left to rightorder.
 12. The system of claim 1 wherein the scale factor is computedfor the closed shape.
 13. The system of claim 1 wherein the scale factoris computed for the size of the text labels in the set of tags.
 14. Thesystem of claim 1 wherein the scale factor is computed for the both theclosed shape and the size of the text labels in the set of tags.
 15. Thesystem of claim 1 further including a printer to print the generated taglayout.
 16. The system of claim 1 further including a communicationsystem to provide the generated tag layout to an online printingservice.
 17. The system of claim 1 wherein there is no overlap betweenthe tags in the generated tag layout.
 18. The system of claim 1 whereinan amount of overlap between the tags in the generated tag layout can beadjusted based on user preference.
 19. The system of claim 1 wherein anamount of overlap between the tags in the generated tag layout can becontrolled using empty boundary space around each tag in the set oftags.
 20. The system of claim 1 wherein the ordering of the tags in theset of tags can be ignored locally to achieve a good fit for thegenerated tag layout.